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ABSTRACT 
One of the primary training tools available to a Jot Commander in Chief (CINC) for 


training his staff on their joint mission essential tasks is a command post exercise 
supported by a computer simulation model. Computer Aided Exercises (CAXs) are an 
essential part of training a component staff, however one weakness with these valuable 
training tools lies in the measurement of the level of training received by the players. In 
most CAXs the players rapidly disperse after the exercise, and little quantitative data are 
captured during the running of the exercise that will allew for quick post exercise analysis. 
This research presents a methodology for evaluating the performance of joint intelligence 
tasks as set forth in the Universal Joint Task List. Instead of attempting to provide 
individual measures for each joint intelligence task, the methodology presented focuses on 
the analysis of significant events that occur during an exercise and relating intelligence 
functions that may have contributed to the ener of such events. Results of 
experimental runs of the Joint Theater Level Simulation are presented to demonstrate the 


methodology and the subsequent analysis process. 
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Executive Summary 


One of the primary training tools available to a Joint Commander in Chief (CINC) for 
training his staff on their joint mission essential tasks is a command post exercise 
supported by a computer simulation model. This is commonly referred to as a Computer 
Aided Exercise (CAX). The primary role of the computer simulation is to present a 
stochastic decision environment within which the staff can be presented with realistic 
results. Based on this simulated environment, staffs implement plans, monitor the current 
situation, and further develop or alter those plans as required by changing requirements. 
Computer Aided Exercises are an essential part of training a component staff, however 
one weakness with these valuable training tools lies in the measurement of the level of 
training received by the players. In most CAXs the players rapidly disperse after the 
exercise, and little quantitative data are captured during the running of the exercise that 
will allow for quick post exercise analysis. Measurement of a staff's capability to perform 
mission essential tasks is ultimately important for two reasons. First, it is important to 
insure that training resources are being used wisely, and progress is being realized in the 
training program. Second, it is important to determine the tasks for which there exists the 
greatest need for further training. This research is intended to furnish a CINC staff with a 
methodology for evaluating the performance of intelligence functions in the context of a 
CAX. 

In the development of this methodology, it is insightful to regard the measure of any 


intelligence process as the answer to the question: How well was the information 


1X 





necessary for optimizing the outcome of an action provided in a timely, accurate, and 
understandable manner? An answer to this question is the goal for any analysis 
methodology. This research provides two methodologies for measuring effectiveness in 
the performance of intelligence joint tasks during a CAX. First, a theoretical methodology 
which relies on a computer simulation model that is currently not in existence is discussed. 
Second, a more practical methodology which could be used with only slight modification 
to some of the computer simulations in use today is presented. The practical methodology 
focuses on the analysis of significant events that occur during an exercise and relating 
causal factors that may have contributed to the outcome of such events. 

This thesis also provides a demonstration of the application of the practical 
methodology using the Joint Theater Level Simulation which is currently in use for the 
training of CINC staffs. Specifically, the demonstration shows how that with only minor 
changes, a simulation model can furnish the data necessary to provide greatly enhanced 
measures of how an intelligence staff performed throughout the conduct of an exercise. 

The methodology presented does not attempt to assign values to the performance of 
each individual joint intelligence task stated in the UJTL, but seeks to determine how 
intelligence functions contributed to the outcome of significant events that occurred during 
the exercise. The methodology is relatively uncomplicated, but retains the robustness 
necessary to be applicable in many different exercise scenarios. Since it is uncomplicated, 
it allows for quick analysis that can be easily understood in post exercise debriefings. An 


additional strength is that interaction with the computer model is very limited with most of 





the data necessary to use the methodology readily available and easy to output in most 


event step simulation models. 








L INTRODUCTION 


A. BACKGROUND 

The Chairman Joint Chiefs of Staff (CJCS) Memorandum of Policy 26 (MOP 26) 
establishes a program for carrying out the joint training responsibilities of the CJCS, the 
Joint Commander in Chiefs (CINCs), and the CINC's component staffs. MOP 26 
institutes a method for identifying training requirements through the review of the CINC's 
mission, and the compilation of the CINC's Joint Mission Essential Task Lists (JMETL) 
required to accomplish that mission. A CINC's JMETL is intended to provide the basis 
for all joint training. 

The Universal Joint Task List (UJTL), a document devised to support the Joint 
Training Manual (MCM 71-92), outlines a comprehensive list of joint essential tasks. The 
UJTL is intended to provide a common language for describing joint warfighting 
capabilities throughout the entire range of military operations to include operations other 
than war. Specifically, tasks are defined as they relate to the strategic (both national and 
theater), operational, and tactical levels of war. Each joint task is broken down into 
supporting tasks which may be further refined into enabling tasks. 

One of the primary training tools available to a CINC for training his staff on their 
joint mission essential tasks is a command post exercise supported by a computer 
simulation model. This is commonly referred to as a Computer Aided Exercise (CAX). 
The primary role of the computer simulation is to present a stochastic decision 


environment within which the staff can be presented with realistic results. Based on this 





simulated environment, staffs implement plans, monitor the current situation, and further 
develop or alter its plan as required by changing requirements. | Computer Aided 
Exercises are an essential part of training a component staff, however one weakness with 
these valuable training tools lies in the measurement of the level of training received by the 
players. In most CAXs the players rapidly disperse after the exercise, and little 
quantitative data are captured during the running of the exercise that will allow for quick 
post exercise analysis. Measurement of a staff's capability to perform mission essential 
tasks is ultimately important for two reasons. First, it is important to insure that training 
resources are being used wisely, and progress is being realized in the training program. 
Second, it is important to determine the tasks for which there exists the greatest need for 


further training. 


B. PROBLEM STATEMENT 
This research develops an exercise analysis methodology for evaluating CINC staff 
performance in the execution of joint tasks during the conduct of a CAX, focusing on 
Strategic Task Three, Develop Theater Intelligence, as stated in the UJTL. Specific 
objectives are : 
1) Develop the analytical tools necessary to provide insight into the value of 
intelligence information during the conduct of a CAX designed to work in 


conjunction with data manipulated by an unspecified computer simulation. 





2) Test methodology using the Joint Theater Level Simulation (JTLS). 

Develop a standardized ASCII file for capturing parameters and demonstrate a 
potential post-exercise analysis. This objective entails a practical application of 
the methodology to an existing theater level simulation. Included in this are the 
alignment of the model's database with required parameters necessary for utilizing 
the methodology, development of algorithms required in post processing, and 
specification of output format. 

It is important to emphasize that this research is part of a larger ongoing research 
project which will attempt to provide an overall analysis methodology for all of the joint 
tasks specified in the UJTL within the context of a CAX. Concurrent with the 
development of the methodology presented in this paper was the development of a 
methodology for assessing the performance of Strategic Task Eight, Provide Theater 
Sustainment, presented in a paper by Captain Ray Combs U.S.A. Since the performance 
of one joint task during a CAX often impacts the performance of another joint task, it is 
strongly recommended that the reader read both papers in order to gain insight into an 
overall analysis methodology which will attempt to identify common causal factors that 


influence significant events that occur during a CAX. 


C. THESIS STRUCTURE 
Chapter II provides a brief overview of the Intelligence Process, and some of the 


known problems in measuring the impact of Intelligence Products. Chapter Ill will 








describe the proposed analysis methodology used to assess staff performance. The 
presented methodology focuses on the analysis of significant events that occur during an 
exercise and related intelligence functions that may have contributed to the outcome of 
such events. Chapter IV looks at applying the methodology to a typical exercise scenario 
using the Joint Theater Level Simulation. This chapter discusses the data manipulation that 
is necessary for post exercise analysis using an existing computer simulation. A summary 
of observations and possible inferences from the validation run are included in this chapter. 


Chapter V summarizes the methodology, and provides recommendations for further 


refinements and study. 








Il. OVERVIEW OF INTELLIGENCE 


This chapter provides a general overview of intelligence. Specifically, it attempts to 
furnish insight into the underlying concepts of the structure of the intelligence tasks in the 
UJTL. In addition, this chapter addresses some of the past problems that have 
necessitated the need for determining the value of intelligence, and the problems that have 


arisen from the attempt to assess that value. 


A. DEFINITION OF INTELLIGENCE 

Intelligence is a term for which no clear-cut definition exists. Not only does it not 
have a precise and agreed definition, but those who attempt precision give it multiple 
meanings. Zhe Department of Defense Dictionary of Military and Associated Terms 
(JP 1-02) defines intelligence as " the product resulting from the collection, processing, 
integration, analysis, evaluation, and interpretation of available information concerning 
foreign countries, or areas." This definition is followed by a listing of forty-one 
subcategories of definitions for intelligence, thus demonstrating the difficulty in containing 
the definition of the term intelligence to one, all-encompassing definition. It is important 
to distinguish the difference between information and intelligence. Information is the raw 
material from which intelligence is derived. To produce intelligence, the raw information 
must be processed into a product which is accurate and relevant. Although not easy to 
define, intelligence does have a clear purpose in that it is intended to provide potentially 


significant information to military planning and operations. Dr. R.V. Jones, the head of 











the British Air Ministry's intelligence section during World War Il, stated, "The ultimate 
objective of intelligence is to enable action to be optimized." [Ref. 1] This statement 
highlights the overall goal for intelligence, that being to reduce uncertainty and the 


element of risk in the planning and execution of any military operation. 


B. LEVELS OF INTELLIGENCE 

Joint Military Intelligence exists at three levels, the highest level being strategic 
intelligence which is required for the formulation of strategy, policy, and military plans and 
operations at the national and theater level. The next level is operational intelligence 
which provides for conducting campaigns and major operations within a theater or area of 
operation. The lowest level is tactical intelligence which supports the planning of battles 
and engagements, focusing at this level on specific combat elements and objectives. These 
three levels of intelligence compose the basic hierarchy of intelligence. Many of the past 
boundaries that existed between these levels of intelligence are growing less clear with the 
changes in information management systems and the rapid increase in technology. As an 
example, satellite reconnaissance, once a tool reserved for strategic intelligence, gradually 
became an integral part of operational intelligence. Now, through such programs as 
Tactical Exploitation of National Capabilities (TENCAP), the development of enhanced 
hardware, software, and communications architecture will allow for direct "sensor to 


shooter" relays of satellite information to the operator at the tactical level. 





C. THE INTELLIGENCE CYCLE 

The most widely accepted model for the intelligence process is referred to as the 
intelligence cycle (Figure 1). The intelligence cycle is divided into four phases. The first 
phase, requirements, involves specifying the perceived need for intelligence and a plan for 
how to best collect the information required to satisfy that need. In the second phase, 
collection, actual assets are tasked with the collection of the information. The third 
phase is the processing of raw information collected into an intelligence product. An 
intelligence product can take many different forms, from a formal National Intelligence 
Estimate (NIE) to a simple verbal report given over the telephone. The final phase is that 


of dissemination of the intelligence product to the user. 


The Intelligence Cycle 





Figure 1. The Intelligence Cycle 











1. Requirements 

In the cycle model of intelligence, requirements are supposed to come from the 
decision makers. However, requirements are usually established in different ways, ranging 
from explicit tasking to vague questions which are left to an intelligence staff to interpret 
into specific intelligence requirements. Intelligence requirements may be generated by 
events that were not anticipated, but because they have taken place, intelligence collection 
is required. Establishment of intelligence requirements is essentially determining what is to 
be done, with the necessary condition of who will do it. Because of the difficulty in 
explicitly defining the inputs for requirement development, the UJTL addresses the 
requirements phase of the intelligence cycle in a very general manner. The joint and 


supporting tasks for the establishment of intelligence requirements are summarized in 


Figure 2. 


Establishment of Intelligence 
Requirements 






ST 3.1 Establish Intelligence OP 5.1 Develop Operational TA 4.1 Develop Tactical Intelligence 
Requirements Intelligence Requirements Requirements 












ST 3.1.1 Evaluate Prior 
Intelligence Requirements 


ST 3.1.2 identify, Prioritize, and 
Validate Intelligence 
Requirements 








ST 3.1.3 Develop Collection Strategy, 
Collection Plan, and Requests 
for information and Products 


Figure 2. The UJTL joint and supporting tasks for intelligence requirements 





2. Information Collection 

Collection in the intelligence cycle refers to the gathering of raw data through human 
sources (special forces, espionage, etc. ), technical means (photography, interception of 
electronic communications, etc.), or in any other manner such as from “open source”. 
[Ref. 2] The raw data to be collected during the collection phase are not restricted to the 
obvious candidates such as enemy unit and target location, but also includes demographics 
of an area of operation which can play a significant role in any military operation. The 
UJTL separates the levels of collection into the joint and supporting tasks depicted in 
Figure 3 . Collection is the phase of the intelligence cycle that comes closest to what is 


most commonly considered intelligence activity. 


Collection of Information 


ST 3.2 Collect Theater Strategic OP 5.2 Collect Operational TA 4.2 Collect Information 
Information Information 


ST 3.2.1 Collect Information on OP 5.2.1 Collect Information 
Enemy Operational 


Strategic Situation, incase 
Geography, Situation and Hazalve TA 4.2.2 Collect Target 


pay lela OP 5.2.2 Collect Information on Information 


Operational Targets 


TA 4.2.1 Collect Information on 
Situation 


ST 3.2.2 Collect Information on 
Theater Stategic Targets 





Figure 3. The UJTL joint and supporting tasks for intelligence collection 








3. Information Processing 

Unfortunately, no matter how good the collected information is, it seldom provides 
the useful information without some analysis or processing. In the vast majority of cases 
the information collected is fragmentary, ambiguous, and susceptible to widely divergent 
interpretations. Thus, the process of analyzing the available information to make 
judgments about capabilities, intentions, and actions of another party is a vital part of the 
intelligence cycle. [Ref. 2] The processing of raw information into intelligence takes a 
wide variety of forms, from the photographic interpreters analyzing imagery data to 
analysts using super-computers to decode enemy communications. Processing of raw data 
into intelligence can take place even on a very low level in the form of a tactical operator 


of a weapon system making basic judgments using past experience. The UJTL breaks 


information processing into the joint and supporting tasks summarized in Figure 4. 


Information Processing 


ST 3.3 Process Strategic Information OP 5.3 Process Operational information TA 4.3 Process Information 
TA 4.3.1 Evaluate Threat 


OP 5.3.1 Evaluate Operational! sefoERetOn 


ST 3.3.1 Evaluate Strategic Threat Threat: Information 


Information 
TA 4.3.2 Evaluate Physical 


OP 5.3.2 Analyze Area of Responsibility Environment 


ST 3.3.2 Analyze Theater Area of / Joint Operations Area 


interest 

TA 4.3.3 Evaluate Social/ 
Political/Economic 
Environment 


OP 5.3.3 Integrate Operational 


ST 3.3.3 Integrate Strategic Intelligence 


Intelligence 


ST 3.3.4 Develop Indications and Cae rons Operational +, 43.4 Integrate Intelligence 


Wamings Information 


OP 5.3.3.2 Develop Operational Target 


ST 3.3.5 Identify Operational nifonuaton 


Vulnerabilities 
OP 5.3.3.3 Identify Enemy Vulnerabilities 





Figure 4. The UJTL joint and supporting tasks for intelligence processing 
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4. Dissemination of Intelligence 

The most commonly overlooked phase of the intelligence cycle is that of 
dissemination. Dissemination is a getting the right intelligence product to the military 
personnel who need it and can use it. Intelligence products can be broken down into three 
categories: current intelligence, basic descriptive intelligence, and intelligence estimates. 
Current intelligence reports are designed to provide intelligence information that may 
affect operations in the immediate future. Included in this category are Indications and 
Warnings, usually referred to as I&W, which are designed to provide timely warning of 
hostile action. 1&W are based on an analysis that when certain "indicators" occur, such as 
a particular airfield conducting air operations, there is a likelihood of enemy action. 

Current intelligence reports may be issued daily, hourly, or immediately upon 
receipt of iis information. Basic intelligence is a general type of intelligence product that 
contains information that is usually less susceptible to change, such as an enemy's air order 
of battle or a hostile country's demographics. Basic intelligence reports are usually issued 
on a periodic basis. Intelligence estimates are the most ambitious type of intelligence 
product. These estimates are supposed to take the broadest view of the subject and 
project the current situation into the future. A substantial effort is devoted to providing a 
single estimate of how a current situation will evolve. However, sometimes dissenting 
opinions are expressed in what has been traditionally called a footnote, even though the 
dissent may be included in the text of an estimate. [Ref. 2] The UJTL breakdown of the 


dissemination of intelligence is summarized in Figure 5. 
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Disseminate Intelligence 


ST 3.4 Prepare and Disseminate OP 5.4 Prepare and Disseminate TA 4.4Prepare and Disseminate 
Theater Strategic Operational Intelligence Intelligence Reports 
Intelligence Reports Reports 

TA 4.4.1 Prepare Reports on 
Target Development 


TA 4.4.2 Prepare Reports on 
Enemy Intention 


TA 4.43 Prepare Reports on 
Tactical Intelligence 


TA 4.4.4 Prepare Reports onEnemy | 
Situation 


TA 4.4.5 Convey Intelligence 





Figure 5. The UJTL joint and supporting tasks for intelligence dissemination 


D. KNOWN PROBLEMS IN MEASURING VALUE OF INTELLIGENCE 
Questions on the value of intelligence are frequently asked, but are seldom answered 
in specific terms. One such example occurs in the weapons system procurement area. 
What is the value of a new intelligence terminal that will allow real time overhead imagery 
to be displayed to a tactical commander? This question can only be answered by 
answering another question. What is the value of real time intelligence in a tactical 
situation? Based on this question many large intelligence agencies such as the National 
Security Agency (NSA) have been asked to justify their budgets measured in the billions 
to Congress and to other supervising bodies. [Ref. 3] In the joint training environment the 
question is asked, How well are the intelligence staffs trained to provide for the success of 


a mission? Once again this a question of the value of intelligence produced by the joint 
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intelligence staff. There is a general consensus that timely, accurate, relevant, and well 
presented intelligence can be seen as force multiplier, and the lack of it can be considered 
both literally and figuratively as a force divider. Unfortunately, there has been no real 
way to quantify the use of intelligence as a force multiplier. Like a catalyst in a chemical 
reaction, intelligence, with its intangible output, is known to be a vital constituent of the 
process it influences so critically, but its precise function is difficult to isolate and measure 
with accuracy. [Ref. 4] 

Frequently, when trying to determine the value of intelligence, the issue of the cost 
of intelligence arises. This issue is not easily addressed, since the determination of the cost 
of the inputs to intelligence products is not always clear, neither 1s it an easy problem to 
attach a value to the benefits from the decisions made with aid of intelligence, as opposed 
to decisions made without intelligence. For example, say an attack submarine has the 
collateral mission of gathering information on enemy shipping activities, and the 
information on enemy shipping supplements information gained from intelligence agents 
and assists a theater commander to make a decision to commence a blockade. How 
would the cost of the submarine collecting information as part of a secondary mission be 
determined? What is the value in dollars of information that merely supplements 
information used in decision making? Unfortunately, cost of intelligence is an area that 
yields little insight into the value of intelligence. 

A methodology for assisting the commander and his staff in assessing the value of 


intelligence shortfalls in its collection, and its contribution to the total campaign during a 
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CAX is described in the next chapter. A demonstration of the application of these 


methods is presented in Chapter IV. 
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I. METHODOLOGY 


In the development of an exercise analysis methodology for evaluating CINC staff 
performance in the execution of joint intelligence tasks during the conduct of a CAX, it is 
insightful to regard the measure of any intelligence process as the answer to the question: 
How well was the information necessary for optimizing the outcome of an action provided 
in a timely, accurate, and understandable manner? An answer to this question is the goal 
for any analysis methodology. This chapter provides two methodologies for measuring 
effectiveness in the performance of intelligence joint tasks during a CAX. First, a 
theoretical methodology which relies on a computer simulation model that is currently not 
in existence is discussed, and second, a more practical methodology which could be used 
with only slight modification to some of the current computer simulations in use today is 


presented. 


A. SIGNIFICANT EVENTS 


A common part to both post exercise analysis methodologies for evaluating 
intelligence staff performance presented is the concept of a significant event. A 
significant event for this paper is defined as any event occurring during an exercise that 
would be useful in the post exercise reconstruction for analysis of intelligence inputs into 
the decision making process. Significant events can be classified in terms of theater, 
operational, and tactical levels as well as to the degree of their significance, which may be 


somewhat subjective. An example of a significant event may be that of an Iraqi invasion 


15 








of Kuwait which would also be an event with a very high degree of overall significance. 
An example of a significant event at the tactical level may be that of an infantry platoon 
engaging in a firefight with the enemy and sustaining minor casualties. The degree of 
significance of this event would have to be evaluated in relation to mission objectives and 
expectations. 

The identification of a significant event allows for the potential determination of 
how well an intelligence staff was able to provide needed intelligence information by 
analysis of intelligence functions and perceptions that had an impact on the outcome of 


the significant events. 


B. A THEORETICAL METHODOLOGY 

A problem with measuring an intelligence staff's performance based solely on the 
outcome of significant events is that it involves the assumption that, provided perfect 
intelligence, all decision making will result in optimal event outcomes. Unfortunately, due 
to multiple factors including the stochastic nature of combat engagements, sub-optimal 
decision making can result in sub-optimal outcomes to significant events, even though 
perfect intelligence information has been provided. 

In order to minimize the impact of imperfect decision making and the stochastic 
nature of most exercise simulations, a methodology for solely evaluating an intelligence 
staffs ability to provide the required intelligence would involve using a computer 


simulation model where the decision makers are part of the simulation, and would make 
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decisions and issue orders according to a programmed set of decision rules. An 
intelligence staff would then provide inputs to the model for one run of the simulation, and 
scenario outcome measures would be collected such as the number of casualties, amount 
of ordinance expended, amount of terrain gained or lost, etc... The outcome measures 
would then be compared to benchmark simulation runs where the model is provided 
perfect information, or no information. An intelligence staffs performance would then be 
compared to the mean outcomes of the two benchmarks, which could be run multiple 
times to assist in the minimization of stochastic effects. 

This type of methodology attempts to sanitize the environment in which 
intelligence products are delivered. A major deficiency in its successful implementation is 
the construction of decision rules. Building a set of decision rules that would handle a 
majority of the situations would be an enormous undertaking. Most military leaders 
would find it extremely difficult to provide the essential elements necessary for them to 
make decisions, or the weighting given to different, sometimes conflicting, intelligence 


reports. 


C. A PRACTICAL METHODOLOGY 
It is unlikely that any simulation in the foreseeable future will provide an 
environment for assessing a CINC's intelligence staff free of nuisance variables. However, 


it is possible to develop a methodology which can work with existing computer simulation 


ey 


models that will provide quantifiable measures that can be used to audit intelligence 
functions leading up to significant events. 

1. Intelligence Report Scoring 

One insightful and manageable question involving the performance of an 
intelligence staff is: How good was the information on the potentially hostile units in the 
area of interest? The concept of report scoring used in this analysis attempts to answer 
that question by taking the approach that an intelligence report on any Other Than 
Friendly Unit (OTFU) can be decomposed into three essential report elements: location 
of the unit, the unit strength, and an estimate of the unit's course of intended action. One 
method for measuring the accuracy of each report element (e.g., position of an enemy 
unit) would be to compare the intelligence reported position to the actual or ground truth 
position and assign a score based on the difference for that particular type of unit. The 
greater the difference between the intelligence report element and ground truth "reality, ” 
the lower the Report Score. 

However, another important factor that must be considered in measuring the 
worth of intelligence is the depreciation of intelligence information value over time. The 
inherent value of intelligence may depreciate with time from : 

Actual changes in the situation. 

Possible, but unknown change - so that the report cannot be used with the same 

level of confidence as before. 


It is important to emphasize that intelligence information need not only be accurate, but 
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that it must also be "fresh", so that a decision maker will remain confident in the use of it. 
The rate at which the value of an intelligence report element will depreciate is a function 
of the type of OTFU reported. For example, intelligence on the location of an infantry 
battalion will depreciate faster than intelligence on the location of an aircraft factory. 
[Ref 1] 

Incorporating the concept that value of intelligence depreciates with time, utility 
weighting curves can be constructed for each OTFU type and intelligence report element 
with respect to time. Utility weighting functions and the resulting utility weighting curves 

express the decay in the value of an intelligence report on an OTFU as the report 1s 
allowed to age. Since the depreciation of intelligence value will vary with report element 
type, the attributes of these utility weighting curves will most likely be different for each 
intelligence report element. 

This paper suggests three possible functional forms, borrowed from the study of 
economics, to express the manner in which intelligence information may be perceived to 
depreciate . Determining the actual depreciation functions for intelligence information on 
any particular unit for a given scenario is challenging, however common properties to all 
intelligence depreciation functions would be that they are monotonic decreasing and they 
return weights between zero and one, with one representing the value of perfect 


information. The first depreciation function is simple linear depreciation: 
w(t) =—-at+1 for Osa<l1, ¢20, (1) 


w(t) = 0 for at > 1, 
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where fis time and ao is the decay rate. A second depreciation function suggested is that 
of "constant percentage" depreciation commonly used in the calculation of depreciation in 
the value of machinery. It could be used for most categories of intelligence information 
and is expressed by: 

w(t) =(1-a)’ for Os a <1, ¢20. (2) 
The third function is essentially one minus a logistic function and is shown below: 

w(t) =1 ~ Tpew for Os a < 1,620,420 . @) 
Equation 3 has the distinguishing property of decreasing slowly for small values of t, and 
then decreasing dramatically faster as t increases, where the shape parameters, « and B, 
adjust the rate of decay. This functional form is intended to model the value of some type 
of information that may retain much of its value for a certain period of time, but after that 
time has passed, it quickly depreciates. These three suggested depreciation functions are 
not meant to capture every possible structure for which information value will depreciate, 
but merely to provide a sound starting point in the development of utility weighting 
functions for the value of intelligence information. Figure 6 provides sample curves for 
the three functions given above with specified decay and shape parameters. 

Currently most computer simulation models do not provide for the real world 
problem of inaccuracies in the reporting of intelligence information. Most simulations 
provide a sensor with a baseline probability of detection, perhaps adjusted for range, and 
environmental variables such as jamming, commonly multiplied by a target detection 


multiplier for the target to be observed. The simulation then performs a draw from a 
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random number generator to determine whether sensor A detects target B. If a detection 


is made, then the simulation provides actual information on the target without error. 
DEPRECIATION IN VALUE OF INTELLIGENCE WITH TIME 
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Figure 6. Depreciation of intelligence value with time. 

There may be some delay time inherent in the simulation for the sensor information to be 
processed, but the information at time of detection is 100% accurate. Consequently, the 
only way that intelligence information becomes inaccurate is by the passage of time. This 
fact results in the determination of the utility for the OTFU intelligence report elements of 
location, estimate of course of action, and strength simply as a function of their age 
without including the possibility that the information has inaccuracies at the time of 
detection. 

Position, course of intended action, and strength information are the core elements 
for an intelligence report on an Other Than Friendly Unit, and examples can easily be 


constructed to show cases where all three elements would be critical to good decision 
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making, as well as examples where knowledge of any one or two elements. would be 
sufficient. Since it may prove difficult to determine the potential critical intelligence report 
element, the Report Score will be composed of a sum of the values of the three elements. 
However, this methodology will still allow that an OTFU's Report Score could be broken 
down into finer detail to provide the audit trail into the values for each individual 
intelligence report element, should that be necessary. 

Typically a decision maker relies on two important pieces of information to make a 
judgment on the quality or value of an intelligence report. The first is reliability of the 
source of the intelligence. However, computer simulations generally do not attempt to 
model unreliable information sources. The second is age of the intelligence which can be 
modeled in most simulations. Therefore the main measure of how good the intelligence is 


on a particular OTFU will be measured by the Report Score shown in equation 4. 
~ Wiis 
Report Score, (t) = —=— (4) 


where 
Wii ~ A utility weighting factor from 0 to 1 of the depreciation of intelligence 
data as a function of intelligence report element type, OTFU type, and 
age. 

indices: 
i - Other Than Friendly Unit { 1st Rep Guard, 2nd Artillery Battalion...} 
t - time {in integer hours from start of CAX t = 0, | ae 
j(t) - age of last intelligence update 


| - intelligence report element type { location, estimate of COA, strength } 
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The Report Score can provide a measure of how effective a CINC's intelligence 
staff was at providing valuable information on OTFUs with only limited assumptions as to 
the structure of the decay of the value of the information as it is allowed to age. 
Combined with the identification of significant events occurring during an exercise, and 
the corresponding significant OTFUs, the Report Score will furnish some insight into the 
ability of an intelligence staff to furnish "fresh" information. 

2. The Asset Needs Function (ANF) 

One of the most demanding functions performed by an intelligence staff is the 
tasking of intelligence assets. Poor intelligence collection asset allocation was specifically 
noted as a deficiency for Operation Desert Shield/Storm. [Ref. 5] In large, complex 
exercises the assessment of the surpluses and shortages of collection assets that occur can 
provide insight into how well an intelligence staff made use of those collection assets at its 
disposal. To adequately address the question of why intelligence information on an enemy 
unit was deficient, it is necessary to determine whether collection assets were not 
available, or were merely under utilized. 

An important = of the problem of collection asset allocation is the 
determination of the potential need for any particular collection asset or type of collection 
asset at any given time. The framework for measuring an intelligence staff's ability to 
adequately provide collection asset coverage within a theater of operation will be centered 
on maintaining a record of each collection asset's availability, and the potential need for 


that asset at any time during an exercise. An intelligence collection asset is considered to 
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be available if it is determined that it could be tasked by the intelligence staff during a 
specified time window to conduct a collection mission. Determination of the potential 
need for any collection asset at a specific time is slightly more involved. Potential need is 
established by whether there exists a significant Other Than Friendly Unit or units that 
have a sufficiently low Report Score, and whether there exists a collection asset that has a 
sufficiently high probability of detection for any of those significant OTFUs. The 
purpose of the Asset Needs Function is to show the existence of a perceived need for a 
particular collection asset to provide information on a particular OTFU at a specific time 
in the exercise. | The Asset Needs Function can be written in the form: 

ANF,,. (t) = (1-(Report Score, (t)) x pd,,(t) x IMF; (t) x SRF, x TRF, (5) 
where the indices are : 

i - Other Than Friendly Unit { Ist Rep Guard, 2nd Artillery Battalion,...} 

k - intelligence collection asset { STARS, TR-1, HUMINT teams,...} 

t -time {in hours from start of CAX t = 0,1,2,....}, 
and the component variables are defined as follows : 

pd,,(t) - the probability of detecting OTFU, i, at time t, given that OTFU, 11s 


within sensor range of collection asset, k . 


IMF (t) - an importance factor assigned for the degree of significance of 
OTFU, i, at time, t. 

SRF, - sensor range factor to adjust for the difference in 
volume of search area covered by the different sensors carried by the 


collection asset, k. 
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TRF,, - a target range factor to compensate for the range of the target from the 
staging point of the collection asset. Essentially, this implies that targets 
at the extreme limits of a collection asset's ability to search may be 
harder to detect. 

For the ANF to return a high value, the Report Score on OTFU, i, must be low, and the 
probability of detection by collection asset, k ,must be sufficiently high. In summary, the 
Asset Needs Function is intended to express the potential of a collection asset to improve 
the Report Score of an Other Than Friendly Unit. 

Once the ANF values have been determined for each collection asset, it is possible 
to classify each by whether there exists a potential need for that asset, and whether it 1s 
available for tasking. Each collection asset at time, t, would be in one of four disjoint 
states: 

State One - there existed a need for the collection asset manifested by a high 

ANF for an OTFU of significance, and the asset was available to be 
tasked. 

State Two - there existed a need for the collection asset manifested by a high 

ANF Report Score for an OTFU of significance, and the asset was 
not available to be tasked. 
State Three - there did not exist a need for the collection asset, and the asset 
was available to be tasked. 
State Four - there did not exist a need for the collection asset, and the asset 
was not available to be tasked. 
Of course, some threshold values will have to be established for an ANF to be considered 
high. 

By identifying significant events that occur during a Computer Aided Exercise, and 
subsequently determining which units were significant to the outcomes of those events, it 
is possible in post exercise analysis to use the Report Scores to gain insight into the ability 


of the intelligence staff to provide the highest value intelligence possible. In addition, by 
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analysis of collection asset availability, and the ANFs for collection assets, it is possible to 


show potential shortages and surplus in the allocation of assets to collect information. 
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IV. APPLICATION 


This chapter demonstrates an application of the practical methodology for evaluating 
the performance of an intelligence staff described in Chapter Ill, using a computer model 
currently in use for the training of CINC staffs. Specifically, it shows how that with only 
minor changes, a simulation model can furnish the data necessary to provide greatly 
enhanced measures of how an intelligence staff performed throughout the conduct of an 
exercise. It is important to emphasize that this experiment was not intended to 


demonstrate tactics or to evaluate the performance of the computer model. 


A. JOINT THEATER LEVEL SIMULATION (JTLS) 

ITLS is an interactive, multi-sided, joint (air, land, naval, and special operations) and 
combined (coalition warfare) constructive simulation model. This computer-based 
wargaming system uses inherent functions - sea, air, land, special operations, intelligence, 
and jowisiies to model conflict (pre-combat, combat operations, and post combat) at the 
operational level of war with tactical fidelity. T he Joint Warfighting Center is the joint 
sponsor of JTLS, and has used JTLS (release 1.85) as the exercise driver for the combined 
exercise Keen Edge 94 in support of U.S. Forces and Japan's Joint Self-Defense Forces, 
and for a combined U.S. Thailand exercise , Cobra Gold 95, among others. 

A new open system version of JILS (release 2.0) which provides a much improved 
user interface was utilized to support this study. The scenario for the study was adapted 


from an existing Desert Storm scenario with orders being scripted to limit operator 
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interaction with the model during the simulation runs. The ability to enter all necessary 
orders before the execution of a simulation run allowed for multiple repetitions of the 
same scenario to be run with different random number seeds. In this experiment the unit 
orders for seven simulation days were entered prior to the first simulation run, and three 
repetitions of the same scenario were run to determine the difference in the output data 
resulting from purely stochastic events. 

Routines for capturing the parameters required to implement the methodology were 
developed by Rolands and Associates Inc., the primary contractor for JTLS. The 
calculation of Report Scores during each simulation run required a file to be maintained 
that sonra the time of each detection, and the time that an intelligence update was 
issued for each Other Than Friendly Unit (OTFU). Although not required for calculating 
Report Scores, it was beneficial to output to the file the mission name that made the 
detection of the OTFU to assist in post exercise analysis. In addition, a separate file was 
maintained which contained the number of each intelligence collection asset available by 
squadron and the time whenever a change in availability status occurred. Both output files 
were in a standard ASCII format that was easily read into a commercial spreadsheet 


package. Sample spreedsheets of both output files are contained in the appendix. 


B. SCENARIO DESCRIPTION 


The simulated intelligence staff for this sample exercise had the assigned mission of 


designing and executing a collection plan for the entire theater of operations. Two areas 


28 








of primary interest were the tracking of Iraqi mobile tactical ballistic missiles (TBMs) and 
the monitoring of the five Iraqi Republican Guard divisions in Iraq at the beginning of the 
exercise. Collection assets were limited to six aircraft types in eight different squadrons 
operating out of four bases in theater. Table 1 provides a summary of the detection 
capabilities of the six aircraft types and the JTLS squadron designations for the squadrons 
that operated them during the exercise. The aircraft types for the experiment were created 
by altering the sensor performance parameters of existing aircraft in the JTLS database. 
The specific purpose of changing the sensor parameters was to assist in exercising the 
analysis methodology, and should not be considered representative of the performance 
characteristics of actual aircraft. Baseline Probability of Detection (BPD) is the baseline 
probability in JTLS that a sensor will detect a target or unit. In JTLS when a target or 
unit is within range of a sensor, the sensors BPD is multiplied by the target's Target 
Detection Multiplier (TDM) and any relevant environmental factors such as jamming or 
bad weather, and the product is the probability of detection of the target by the sensor. 
The Range Factor (RF) was determined by the range of the sensor which for airbome 
sensors is measured in hexes. Therefore, for this analysis, if a sensor has a range of two 
hexes, it would have a RF of two. Mean Fusion Time (MFT), which is measured in hours, 
is the mean of an exponential distribution of how long it takes from sensor detection to 
delivery of the information to the player. If an aircraft has a MFT of zero, it is considered 


to be a real time sensor. 


29 








Range Factor |Operating Total Number 
(RF) Squadrons of Aircraft 





= 


Table 1. Summary of Reconnaissance Aircraft Types 
The collection plan developed called for the theater to be divided into three major 


sections. The first section is the portion of Iraq where the TBM batteries would be forced 
to operate in order to launch attacks. The second section is a 300 km wide corridor from 
Baghdad to the Kuwait border. The third section is Kuwait, and a zone 100 km into Iraq 
along the Kuwaiti border. These three major sections were patrolled by the use of eight 
different air reconnaissance routes for each section. 

An Air Tasking Order (ATO) for each day would task squadrons to fly missions at 
designated times along specified routes. Utilizing the JTLS squadron designations, 
squadrons 94FS, 389FS, and 152ANG were assigned to the TBM section patrols, while 
VF-154 patrolled the corridor to Baghdad, and the remaining squadrons patrolled Kuwait 

and the Iraqi border. It is important to emphasize that although the JTLS squadron 
designations are used throughout this application no conclusions should be drawn as to the 


capabilities or characteristics of actual squadrons. All missions were composed of two 
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aircraft. A total of 144 missions (288 sorties) were scheduled to be flown on each of the 


seven days of the exercise. 


C. ANALYSIS OF TBM INTELLIGENCE 
The intelligence staff for the simulated exercise allocated three squadrons for a total 

of 60 aircraft for the task of reporting the eight different mobile TBM batteries which 
were expected to be operating in a 200 km x 200 km area. The collection plan was that 
each squadron would be responsible for flying missions composed of two aircraft every 
four hours along predetermined search routes each day. The goal of this collection plan 
was to provide the most up-to-date information possible on each of the eight mobile TBM 
batteries. 

1. Significant Events 

In an actual exercise, critical events would be determined for post exercise analysis. 
Examples of possible critical events in this exercise would be a massive launch of TBMs 
on a specific day, or the random launches of TBMs over the entire period of the eXxe§cise. 
To facilitate the demonstration of the analysis methodology for the case of the mobile 
TBMs, the missile launches will be considered to be randomly distributed throughout the 
exercise, thus the entire time span of the exercise is considered to be one significant event. 

2. Report Scores 

In the calculation of Report Scores, the first question that must be considered 1s: 


What is the expected decay rate and functional form for representing the depreciation of 
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intelligence for the Other Than Friendly Unit (OTFU) reported? For this case of the 
reporting of TBM batteries, only the location information will be considered, since 
strength and course of intended aeion information would not be expected to change 
significantly in the limited exercise scenario considered. A base depreciation function for 
TBM batteries over the course of the exercise is the constant percentage depreciation 
function shown in equation 2 with a decay rate of 0.2 per hour (20% per hour) for utility 
weights used in the Report Score for specific TBM batteries. Two alternative 
depreciation functions are presented to show the sensitivity of the Report Score to 
changes in the functional form and the decay rates of the depreciation function. The 
alternative depreciation functions are a constant percentage depreciation with a decay rate 
of 0.1 per hour (10% per hour), and the logistic functional form of equation 3 with shape 


parameters a=0.8 and B=100. Figure 7 shows the three depreciation functions described 


above. 


DEPRECIATION IN VALUE OF INTELLIGENCE WITH TIME 


w(t)=1-1/(14+100exp(-0.81) 


w(t)=(1-0.2)t 
wit}=(1-0.1)*t 


time in hours 





Figure 7. Utility weight functions used in analyzing TBM intelligence Report Scores 
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Utilizing the base constant percentage depreciation of 20% per hour Figures 8 and 9 
show the Report Scores for two of the eight TBM batteries for the seven simulation days 
of repetition one. Each of the "peaks" in the plots corresponds to an intelligence update 
resulting from a sensor detection of the TBM battery, and the "valleys" correspond to the 
depreciation in value of the updates as they are allowed to age. Note that some of the 
"peaks" in the Report Score do not start at one, because the fusion time inherent in some 
sensors results in a delay in the time it takes for an intelligence update to be delivered from 


the time of detection. 
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Figure 8. Report Score for TBM battery A 
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REPORT SCORE FOR TBM BATTERY G 


i 
it 
oO 
© 
” 
~ 
a 
Oo 
Q. 
uJ 
~ 





Figure 9. Report Score for TBM battery G 

If the random launches of TBM over the entire seven day exercise were regarded as 
being significant events, then it would be of interest as to how well the TBM batteries — 
were being detected over that total period. Figures 8 and 9 indicate where there existed 
long periods of low Report Scores. One area of particular interest is the period around 
hour 80 ( the fourth day) in which Report Scores for both TBM batteries were basically 
zero for almost an entire day. In fact, the Report Scores for all eight TBM batteries 
indicate low Report Scores around the fourth day. The low Report Scores on the fourth 
day of the simulation would indicate that closer analysis should be done into the potential 
causal factors for the reduced detections, and how they relate to the overall campaign 
plan. Some causal factors for day four were that Aircraft A, the best sensor for detecting 
TBMs, was tasked with patrolling less prolific search routes, and was suffering from lower 
aircraft availability due to maintenance. An important note is that in this limited 


demonstration exercise no attrition was played for either collection assets or TBM 
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batteries. If attrition had been played, then the campaign plan may have called for 
prosecution of the mobile TBM batteries early, and the collection plan may have been 
adequate. However, if the early attacks were unsuccessful then the window of low Report 
Scores on day four may indeed denote a problem. 

Figure 10 shows a graphical representation of how the Report Score for TBM 
battery A would appear using the three different depreciation functions given in Figure 7. 
Although three different depreciation functions were used, Figure 10 shows the same 
pattern of low Report Scores on day four. The differences between the Report Scores 
utililizing slightly different depreciation functions can become more pronounced by the 
selection of a threshold value, perhaps representing a minimum acceptable level, and 


calculating the amount of time spend above or below this threshold. 
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Report Score for TBM Battery A Using Constant Percentage Depreciation (0.8) 
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Figure 10. Report Scores for TBM Battery A using three different depreciation functions 
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3. Asset Needs Function 

From Table 1, it can be seen that of the three types of aircraft tasked with locating 
the mobile TBM batteries, Aircraft Type A had the greatest potential for detection with 
BPD of 0.5 and SRF of two. Consequently, for demonstration of the Asset Needs 
Function (ANF) Aircraft Type A will be the sample asset. The ANF defined in Chapter III 
will be simplified for this presentation by holding all factors constant with the exception of 
Report Scores for each OTFU. A value of 0.8 was assigned to the Target Detection 
Multiplier (TDM) for the mobile TBM batteries, hence the probability of detection for 
Aircraft Type A was 0.4, which is BPD x TDM. The assumptions for the ANF 


components for Aircraft Type A and mobile TBM batteries used are summarized below: 


pd = 0.4, 
IMF =1, 
SRF=2, 
TRF=1. 


Thus the ANF for k = Aircraft Type A, with respect to i= mobile TBM batteries, at 
time, t , 1S: 
ANF, , (t) = (1-Report Score; (t))x04x2. 
In this simplified case the ANF is almost the compliment of the Report Score, so a "peak" 
in the Report Score will result in a "valley" in the ANF, and vice versa. Note that this 
should not always be assumed to be the case, since a decrease in Report Score may be the 
result of a factor that contributes to a decrease in the ANF as well. For example, heavy 


jamming or extremely bad weather may render a particular sensor completely ineffective 








against an OTFU, so both the Report Score and the ANF for that OTFU will decrease, as 
the collection asset's detection capabilities are degraded. 

One potential use of the ANF is to group the exercise into time intervals (e.g., 
days), and compute daily average ANFs for a particular asset. Table 2 shows such an 


example for Aircraft A, and the mobile TBM batteries. 
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Table 2. Average Daily ANF Values for Aircraft A and TBM Batteries. 


By using the daily averages in Table 2 the ANF for a particular collection asset and a 
set of OTFUs, it is possible to determine which time periods exhibited the greatest need 
for the collection asset, and to prioritize the requirement for the collection assets to track 
specific OTFUs . Note the relatively small amount of variability between the ANFs for the 
TBM batteries on any given day. Also note that days four and seven had the largest ANF 
in comparison to the other days of the scenario. Using daily average ANFs has the 


additional advantage of allowing for the comparison of whether the collection asset was 





available or not on any particular day. One way to determine asset availability is by 
calculating the average number of aircraft available at any time throughout the day. By 
comparing average collection asset availability with average ANFs, it is possible to show 
potential surpluses and shortages of the asset. For example, examination of Table 2 
reveals high average ANF's for Aircraft A and all TBM batteries on days four and seven, 
but comparison to Table 3 shows on average only two of Aircraft A were available in 
theater. With an average of only two Aircraft A available it is doubtful that any additional 
sorties could have been generated that potentially would have improved the reporting of 
the mobile TBM batteries. On day seven, however, there existed an average of 9.9 
Aircraft A available throughout the day, which may have allowed for the tasking of 


additional sorties which may have resulted in an improvement in the reporting of TBM 


Ls Average # of 
Aircraft A 
Available 
por | 99 


Table 3. Average Number of Aircraft A Available on Each Day of the Exercise 


batteries. 
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Once preliminary analyses of the ANFs for critical OTFUs and collection assets have 
revealed time intervals in which there existed high need for a collection asset, as on days 
four and seven, investigation into possible causal factors can be pursued. Investigation 
into causal factors for the case of mobile TBM batteries should start by analyzing air 
tasking in the daily ATOs, then searching for common features of days with low asset 
need and days with high asset need for particular collection assets. In addition, asset 


availability should be analyzed looking for trends that may have resulted in periods of high 


ANEs. 


D. ANALYSIS OF REPUBLICAN GUARD INTELLIGENCE 
The intelligence staff for the simulated exercise was tasked with the surveillance of 

the largest perceived enemy threat in the theater, the five Iraqi Republican Guard Divisions 
held in reserve at the start of the scenario. The Republican Guard Divisions were 
expected to be highly mobile and capable of quick entry into any engagement. Intelligence 
was limited in that only Aircraft D had the performance characteristics necessary to 
operate in the region where the Republican Guard Divisions were expected to be located. 
This limited the efforts to track the five divisions to the two aircraft squadrons equipped 
with Aircraft D, a total of 20 aircraft. 

1. Significant Events 

On the first day of the scenario, Iraqi forces in Kuwait attack into Saudi Arabia. 


This invasion is considered to be a significant event in the exercise. A key area of interest 
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for post exercise analysis of the performance of intelligence functions is how well the Iraqi 
Republican Guard Divisions were tracked on the first day of the scenario. This part of the 
application of the methodology is intended to illustrate the potential analysis of a one day 
time period out of a larger exercise time frame. 

2. Report Scores 

Once again, the first question in the calculation of the Report Score for an OTFU 
that must be considered is: What is the expected decay rate and functional form for 
representing the depreciation of intelligence information for that OTFU? For this case of 
the five Republican Guard Divisions, a base depreciation function is the constant 
percentage depreciation function expressed in equation (2) and illustrated in Figure 7 with 
a decay rate of 0.2 per hour (20%). Also, as in the TBM example, only the location 
information will be considered. Figures 11 through 15 show graphical representations of 
the value of intelligence for the five Republican Guard Divisions throughout the first day. 
It is important to point out that Aircraft D, which was the only aircraft type tasked with 
the reporting of the Republican Guard units, was equipped with a non-real time sensor 
with a MFT of 1.0 hours. The MFT results in a delay from the time of detection of a unit 
by the sensor to the time of a report being issued. Thus, when the report is received the 
intelligence information has already been allowed to age. Because of this delay from 
detection to reporting, and the decay rate of 0.2 per hour, the average Report Score for a 
Republican Guard Division at the time of a report is 0.8, instead of the 1.0 that would be 


expected of a real time sensor. Also, since MFT is the mean of an exponential distribution 


4] 





sampled by the computer model to determine the actual fusion time from detection to 
reporting, it is not uncommon to have reports being issued in excess of three hours after 
detection occurred. This is evident in Figure 11 where the first report issued at the four 
hour point is from a detection almost three hours prior. 

Analyses of the Report Scores for the five Republican Guard divisions during the 
first day of the scenario reveal some interesting points. First, Figure 12 shows that no 
reports were issued on the 10th Armor Division until the 15 hour mark. Also note that 
both the 52nd Armor and the Baghdad Divisions suffered from low Report Scores during 
the first 15 hours of the exercise. Second, the Report Scores for all five Republican Guard 
Divisions demonstrate the disadvantage of a non-real time sensor compared to what would 
have been expected of a real time sensor in that the time spent between detection and 


reporting results in a much lower average Report Score over the time period. 


Day One Report Score for Hammurabi Division 
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Figure 11. Day one Report Score for Iraqi Republican Guard Hammurabi Division 
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Day One Report Score for 10th Armor Division 
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Figure 12. Day one Report Score for Iraqi Republican Guard 10th Armor Division 


Day One Report Score for Baghdad Division 
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Figure 13. Day one Report Score for Iraqi Republican Guard Baghdad Division 
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Day One Report Score for 52nd Armor Division 


Report Score 


Time in Hours 





Figure 14. Day one Report Score for Iraqi Republican Guard 52nd Armor Division 
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Figure 15. Day one Report Score for Iraqi Republican Guard 17nd Armor Division 
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E. SUMMARY 

This chapter has demonstrated through the use of smail, illustrative examples how the 
methodology can be applied to a proven simulation model. The methodology is intended 
to provide for post exercise analysis of intelligence functions through the identification of 
significant events and the use of measures which attempt to capture the value of 
intelligence for key units. In addition, the methodology shows the potential for improving 
the intelligence picture of the battlespace by looking at the availability and need for 
collection assets. This experiment, conducted with JTLS, has established how, with only 
limited interaction with the model, all of the information necessary to implement the 


methodology can easily be output. 
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V. SUMMARY AND RECOMMENDATIONS 


A. SUMMARY 

This research has provided a post exercise analysis methodology for evaluating the 
performance of intelligence functions during a computer aided exercise. The methodology 
presented does not attempt to assign values to the performance of each individual joint 
intelligence task stated in the UJTL, but seeks to determine how intelligence functions 
contributed to the outcome of ra ee events that occurred during the exercise. The 
first step in the implementation of the methodology is to determine the significant events. 
Next, determine how well the intelligence staff was able to provide detection and 
subsequent updates of the Other Than Friendly Units of interest throughout the exercise 
by analysis of unit report scores. Lastly, by analysis of Asset Need Functions for critical 
units during the exercise and collection asset availability, determine where there may have 
existed potential shortages, and surpluses, in allocation of assets. 

One strength of the methodology is that it is relatively uncomplicated, but retains the 
robustness necessary to be applicable in many different exercise scenarios. Since it is 
uncomplicated, it allows for quick analysis that can be easily understood in post exercise 
debriefings. An additional strength is that interaction with the computer model is very 
limited with most of the data necessary to use the methodology readily available and easy 


to output in most event step simulation models. 


47 








B. RECOMMENDATIONS 

A fundamental step in the implementation of the methodology is the determination of 
the depreciation functions for intelligence information described in Chapter III. Through 
the use of surveys, simulation, and historical analysis it is possible to develop a library of 
depreciation functions which correspond to the intelligence report elements for specific 
unit types (e.g., position information on mobile TBM batteries). Such a library would be 
essential in the development of a packaged post exercise processor capable of rapidly 
processing the large quantities of data generated in an actual exercise. 

Theater level models by design usually lack tactical fidelity, and J TLS is no exception. 
For this reason, the JTLS database was modified to provide collection assets with 
reasonable capabilities of what might be expected for the aircraft types described in 
Chapter IV. To truly evaluate intelligence functions, especially the use of specific 
collection assets, the JTLS database would need to be modified to present a more accurate 
representation of actual collection assets and sensor loads capabilities. 

JTLS also does not attempt to model uncertainty or inaccuracy in the reporting of 
target information upon detection. As discussed in Chapter III, as with many theater 
simulation models, the information on the target is perfect at the time of detection. 
Although modeling imperfect detection information is a challenging task, one candidate 
method would be to perturb the detection information depending on the detecting sensor 


and the target, and then assign confidence levels with the report of the information to the 


player. 
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The development of a post exercise processor to automate calculations necessary to 
furnish Asset Needs Functions and Report Scores would offer the ability to quickly sort 
information necessary for analysis. Chapter IV of this research demonstrated how a 
methodology for evaluating intelligence functions can be implemented in a small 
experimental exercise. Real world exercises commonly consist of hundreds of players and 
can run over a period of weeks. To provide timely post exercise analysis of the potentially 
vast quantity of data, it is essential that an automated post exercise processor be 
developed. The post exercise processor should be contain a graphical interface capable of 


generating a host of tables and charts to facilitate timely analysis and debriefing of results. 
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APPENDIX 


A. JTLS OUTPUT FILE FOR UNIT DETECTIONS 
The report below is a sample of the JTLS output data imported directly into 
Microsoft Excel for the detection of mobile TBM batteries in the experimental exercise 


discussed in Chapter IV. 
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B. JTLS OUTPUT FILE FOR COLLECTION ASSET AVAILABILITY 
The report below is a sample of the JTLS output data imported directly into 
Microsoft Excel for the availability of collection assets in the experimental exercise 


discussed in Chapter IV. 
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